Cognitive walkthrough - an element in system development and evaluation:experiences from the eWALL telehealth system

AbstractEpidemiological changes in the population lead to an increasing number of elderly people with a chronic disease. Telehealth is proposed as one of the solutions for the growing challenges of the health care system caused by these changes. The telehealth system eWALL seeks to promote the independent living of people with chronic obstructive pulmonary disease, mild cognitive impairment or age related impairments. The eWALL system is in a developmental stage in which partners from 14 different European countries are included. A three phase cognitive walkthrough-approach was performed on the eWALL system in order to evaluate the usability of the system. First the cognitive walkthrough performed by experts, second, rating of the identified usability problems identified by other medical partners, and third, discussion on a plenary telecommunication call among medical partners and technical partners. (n=119) usability problems were identified distributed among the 14 functionalities of the telehealth system. The majority of the usability problems were discovered in the functionalities: ‘TV’ (n=21), ‘Calendar’ (n=20), and ‘Environmental box’ (n=18). The least usability problems were identified in the functionalities: ‘My sleep’ (n=1), ‘Photo frame’ (n=2), and ‘My Everyday Life’ (n=3).The results of the cognitive walkthrough served as a concrete, structured and constructive collaborative tool between the medical partners and the technical partners involved in the eWALL project

An Iterative, Mixed Usability Approach Applied to the Telekit System from the Danish TeleCare North Trial

Objective. The aim of the present study is to evaluate the usability of the telehealth system, coined Telekit, by using an iterative, mixed usability approach. Materials and Methods. Ten double experts participated in two heuristic evaluations (HE1, HE2), and 11 COPD patients attended two think-aloud tests. The double experts identified usability violations and classified them into Jakob Nielsen’s heuristics. These violations were then translated into measurable values on a scale of 0 to 4 indicating degree of severity. In the think-aloud tests, COPD participants were invited to verbalise their thoughts. Results. The double experts identified 86 usability violations in HE1 and 101 usability violations in HE2. The majority of the violations were rated in the 0–2 range. The findings from the think-aloud tests resulted in 12 themes and associated examples regarding the usability of the Telekit system. The use of the iterative, mixed usability approach produced both quantitative and qualitative results. Conclusion. The iterative, mixed usability approach yields a strong result owing to the high number of problems identified in the tests because the double experts and the COPD participants focus on different aspects of Telekit’s usability. This trial is registered with Clinicaltrials.gov, NCT01984840, November 14, 2013